feat(local): replace Ollama with a bundled llama.cpp runtime, a curated GGUF catalog, and a hardware-aware fit estimator - #1819

Phase 7 now replaces the Ollama sidecar with a packaged llama-server
and stops shipping llmfit entirely. Hardware comes from
ggml_backend_dev_memory() via ctypes; scoring moves offline to a
maintainer machine and ships as data in curated-models.json.

- 07-llamacpp-runtime: add Contracts, authoring and appendix sections;
  pin one quantization per entry, fix context at load, defer manifest
  growth and speed gating out of v1
- 05c-packaging: replace the llmfit section with llama-server staging,
  split the catalog and Hugging Face search routes, make .gguf import
  a first-class offline path
- 05b-connections: rename provider ollama -> llamacpp, note revision
  0012 rebuilds selected_models for the CHECK constraint
- 00c-data-model: rename egress destinations to model_download and
  model_search
- 00b-diagrams, frontend/01-shell, frontend/05-install-ux: follow the
  same rename and drop the scan step from the install flow

…d llmfit

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

… Ollama

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…ettings

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…r remote files

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…gh ggml

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

… model

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…-compatible provider

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…s, and thinking control

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…d Hugging Face search

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…on module

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…provider

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…el_search

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…ision 0012

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…oken

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

… runtime

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…g Face

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…time

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…on module

…target

…tures

…t path

…d guess

Three failure modes existed even at the old 16384 floor:

- A prompt that outgrew the window surfaced as "provider temporarily
  unavailable" instead of the true, retry-proof cause. Classify
  llama.cpp's own exceed_context_size_error as its own ChatErrorKind
  with copy that tells the user what actually happened.
- The answer had no max_tokens, so a reply could run into the window's
  edge and stop mid-sentence with no error at all. Reserve tokens for
  it up front and cap the request.
- History was trimmed against a hardcoded 3000-token constant that had
  no relationship to the model's actual n_ctx. Derive the budget from
  the real window instead, read once via a new context_tokens() on the
  Generator protocol (llama.cpp answers it from /props; a remote
  endpoint honestly returns None and keeps today's fixed fallback).

CONTEXT_FLOOR_TOKENS drops from 16384 to 8192, and plan_load now widens
across a fixed set of rungs (8192/16384/32768) instead of an unbounded
binary search, so the catalog badge, the preset, and the actual load
can only ever agree on one of a small, testable set of windows. 8192
is only safe because of the history-budget fix above: a constant sized
for 16384 would have silently overflowed the narrower floor.

…ops per load

Two related fixes to the local runtime seam:

- Generator.token_count() reads llama.cpp's /tokenize endpoint, so
  history trimming uses the model's real tokenizer instead of a
  ~4 chars/token heuristic that could under-count dense text by
  15-20%. A remote endpoint has no equivalent, so it keeps the
  heuristic exactly as before.
- LlamaCppProvider.capabilities() now caches its /props read per
  model instead of fetching it fresh on every message. context_tokens()
  and chat()'s own template shaping were each issuing an independent
  /props round trip on every single turn; measured live, that showed up
  as real, avoidable latency layered onto an already slow load. The
  cache is cleared the one time it can go stale: _ensure_loaded()
  issuing a fresh load.

…ication

A build's badge and its star eligibility were computed by two
independent thresholds over the same offload_fraction: copy.py drew
lines at 0.25/0.5 to choose wording, recommendation.py drew a separate
line at 0.75 (MAX_OFFLOAD) to choose whether to star it. Nothing kept
them in step, so a build could be starred while its own badge read
"Too big for the GPU, so part runs on the CPU" or worse.

New modules/llm/fit/speed.py classifies a verdict into one SpeedTier
once; copy.py renders wording from the tier instead of its own bands,
and recommendation.py's eligibility is RECOMMENDABLE_TIERS (FULL and
LIGHT_SPILL) on that same tier. MAX_OFFLOAD is gone. A recommended
build can now only ever carry "Full speed" or "Most of it still fits"
wording, by construction.

test_badge_matches_the_load.py gains a property test sweeping every
curated model against every budget shape, asserting a starred row's
badge is never spill wording. Three test_recommendation.py fixtures
tuned to the old 0.75 cutoff are recomputed against the tighter one,
one of them (the module's founding case) with numbers that had gone
stale independent of this change.

The gate was two lists answering one question: a 152 name table generated
from the pinned libllama, and a 21 name exclusion set beside it. The table
was the only thing that could tell "llama.cpp has no builder for this yet"
from everything else, and it cost a script run at every pin bump, a datas
entry, and a file that had to ship on Windows because the symbol it reads
is not exported there.

Measured against the 1000 most downloaded GGUF repos, it bought less than
it looked like. Dropping it lets 18 of 979 install and then fail, all of
them chat architectures released in the last few weeks. That is accepted:
curated models are the default path, search is opt in, and the failure now
says so instead of telling the reader to try again shortly.

What replaces it is one denylist, grouped by what a model is rather than
by which fact named it, so a GGUF architecture and a repo pipeline tag
share the table. The tag exists because a header can be honest and still
mislead: an embedding model declares mistral3, a voice model declares
qwen3, a reranker declares qwen3. Refusing those architectures would
refuse Mistral and Qwen, so only the tag catches them. It rides along on
the listing call read_repo_facts already makes.

Measured over the same 1000 repos: 75 installs that fail become 18, with
zero refusals of a model that would have worked. The tags that mean chat
are held out by a test, since image-text-to-text alone is 28% of admitted
repos and denying it would empty the search screen.

scripts/audit_gate.py replaces the guard the generated table used to give.
A test held every denylist key against the names llama.cpp defines, which
is how granite-moe, granite-hybrid and nemotron-h were caught refusing
nothing. With no table, a misspelling now shows up in the audit as an
entry that matched no repo in the sample.

…summaries

One denylist replaces the generated architecture table and its script. The
gate reads the candidate build's header instead of Hugging Face's repo
summary, which parses one file per repo and reported a chat model shipped
beside a vision sidecar as `clip`.

Measured over the top 1000 GGUF repos: wrong refusals 17 to 0, installs that
then fail 75 to 18, and those now say so instead of asking the reader to retry.

_ensure_loaded read /models then posted /models/load, losing to the request
already loading the model: 400 `model is already running`, which failed title
generation. The proxy autoloads, so the call was a duplicate.